Better together: A multistakeholder approach to developing specialty-wide entrustable professional activities in emergency medicine.

Purpose: Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care. Methods: In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback. Results: Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally. Conclusions: To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs.

Help Me Help You: How to Request a Letter of Recommendation.

Letters of recommendation (LORs) are an essential component of a career in medicine. The process for obtaining certain letters, particularly those associated with scheduled periods of professional transition, often is governed by established institutional or specialty norms. The process of requesting LORs in more common scenarios-local or national awards, committee assignments, and leadership positions-many times is less clearly defined, however. Despite the important role that LORs play in professional development, the published literature on how to solicit a recommendation is limited, creating challenges for both those requesting LORs ("applicants") and the letter writers. This perspective piece offers insight on how to best identify and communicate with a potential writer. These suggestions are derived from the limited relevant literature and from the authors' experience both with requesting letters themselves and writing letters as leaders in undergraduate and graduate medical education. The goal is to reduce ambiguity for applicants and ensure that writers receive the information necessary to provide an informed and effective recommendation.

Clinicians' Insights on Emergency Department Boarding: An Explanatory Mixed Methods Study Evaluating Patient Care and Clinician Well-Being.

BACKGROUND: The aim of this study was to describe clinicians' insights into the quality and safety of patient care delivered to emergency department (ED) boarding patients, as well as clinician safety and satisfaction related to ED boarding. METHODS: This was a single-site, mixed methods sequential explanatory study. Quantitative data were obtained from a cross-sectional survey sent to ED attending physicians, resident physicians, advanced practice providers, and nurses. Semistructured focus group interviews with a subsample of participants sought to add depth to the interpretation of survey data and identify areas of improvement in boarding care. Chi-square and Wilcoxon rank sum tests were used to evaluate for response differences between groups. Qualitative data were thematically coded and analyzed. RESULTS: A total of 94 questionnaires were obtained for a response rate of 34.1%. Clinicians reported that boarding highly contributed to the perception of burnout. All groups reported high rates of perceived verbal and/or physical abuse from boarding patients (86.8% of nurses, 41.1% of providers, p = 0.0002). A total of 39 clinicians participated in focus groups regarding boarding care, and six themes were identified, including patient safety concerns, lack of knowledge/resources/training, and poor communication. Key themes identified as possible solutions to improve care included standardization of care, proactive planning, and culture change. CONCLUSION: Clinicians identified many concerns regarding patient safety and the quality of care delivered to boarding patients and identified several areas for improvement. Clinicians also felt that boarding negatively affected their satisfaction and safety.

Implementation of a standardized pregnancy screening process to address gender disparities in radiology turn-around-time and ED length of stay.

OBJECTIVE: The objective of this initiative was to quantify and intervene upon suspected gender disparities in CT turn-around-time and emergency department (ED) length of stay. METHODS: This was a single-site before-after quality improvement initiative including patients aged 12-50 who underwent CT chest and/or abdomen/pelvis. The intervention included protocolization of the pregnancy screening process in triage. Primary outcomes included the difference between women of childbearing age and similarly aged men in regards to CT turn-around-time and ED length of stay. Pre- and post-intervention data were analyzed, including an "intensive intervention period" subanalysis. RESULTS: CT turn-around-time for women of childbearing age was 19 min longer than for similarly aged men at baseline and did not change significantly post-intervention. ED length of stay was 27 min longer for women of childbearing age compared to similarly aged men at baseline and 7 min longer post-intervention, although this was still a significant difference. During the intensive intervention period, CT turn-around-time for women of childbearing age was 15 min longer than similarly aged men but the difference in ED length of stay of 10 min was no longer significant. CONCLUSIONS: There is gender disparity in CT turn-around-time and ED length of stay in our ED, highlighting an important area for improvement to promote equitable care. A quality improvement initiative that aimed to protocolize pregnancy testing in triage did not show sustainable improvement in these outcomes but did result in increased pregnancy testing.

RéSUMé: OBJECTIF: L'objectif de cette initiative était de quantifier et d'intervenir sur les disparités présumées entre les sexes dans le délai d'exécution du scanner et la durée du séjour au service des urgences. MéTHODES: Il s'agissait d'une initiative d'amélioration de la qualité à un seul site avant-après, incluant les patients âgés de 12 à 50 ans qui ont subi une TDM thoracique et/ou abdominale/bassin. L'intervention comprenait la protocolisation du processus de dépistage de la grossesse lors du triage. Les résultats primaires comprenaient la différence entre les femmes en âge de procréer et les hommes du même âge en ce qui concerne le délai d'exécution du scanner et la durée du séjour aux urgences. Les données avant et après l'intervention ont été analysées, y compris une sous-analyse de la "période d'intervention intensive". RéSULTATS: Le temps d'exécution de la TDM chez les femmes en âge de procréer était 19 minutes de plus que chez les hommes de même âge au départ et n'a pas changé de façon significative après l'intervention. La durée de séjour aux urgences était de 27 minutes de plus pour les femmes en âge de procréer par rapport aux hommes du même âge au départ et de 7 minutes de plus après l'intervention, bien que cette différence reste significative. Au cours de la période d'intervention intensive, le temps de passage au scanner des femmes en âge de procréer était de 15 minutes plus long que celui des hommes du même âge, mais la différence de 10 minutes dans la durée de séjour aux urgences n'était plus significative. CONCLUSIONS: Il existe une disparité entre les sexes en ce qui concerne le délai d'exécution du scanner et la durée de séjour dans nos urgences, ce qui met en évidence un domaine important à améliorer pour promouvoir des soins équitables. Une initiative d'amélioration de la qualité visant à protocoliser le test de grossesse au triage n'a pas montré d'amélioration durable de ces résultats mais a entraîné une augmentation des tests de grossesse.

Implementation and Effects of a Team-Based Quality Improvement Training Program Across a Health System: The Northwestern Medicine Academy for Quality and Safety Improvement.

BACKGROUND: Professional organizations emphasize the need to train health care professionals in quality improvement (QI). Many reports of QI education programs involve small numbers of participants. Little is known about QI education programs on a larger scale and whether participants subsequently engage in QI activities. METHODS: The Northwestern Medicine Academy for Quality and Safety Improvement (NM AQSI) was developed to prepare individuals across the Northwestern health system to lead QI. The 7-month program consists of classwork and team-based project work. Participant knowledge was assessed using a multiple-choice test and adapted Quality Improvement Knowledge Application Tool (QIKAT). The study team surveyed participants 18 months after AQSI completion to assess their activity in QI. Project status was assessed at AQSI completion and at 18 months. RESULTS: Over 8 years, 80 teams consisting of 441 individuals participated, representing a range of specialties, settings, and professions. Participants had higher multiple-choice test (70.7 ± 14.0 vs. 78.1 ± 13.0; p < 0.001) and adapted QIKAT scores (56.1 ± 15.9 vs. 60.8 ± 15.8; p < 0.001) after AQSI. The majority of participants at 18 months (180/243; 74.1%) had engaged in subsequent QI efforts; many (105/243; 43.2%) had led other QI projects, and (103/243; 42.4%) provided QI mentorship to others. The majority of teams (53/80; 66.3%) improved project measure performance. CONCLUSION: NM AQSI is a team-based QI training program that shows measurable improvements in care and a high degree of participants' subsequent involvement in QI. Other health systems may use a similar approach to successfully train health care professionals to lead QI.

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy.

INTRODUCTION: Emergency department thoracotomy (EDT) is a lifesaving procedure within the scope of practice of emergency physicians. Because EDT is infrequently performed, emergency medicine (EM) residents lack opportunities to develop procedural competency. There is no current mastery learning curriculum for residents to learn EDT. The purpose of this study was to develop and implement a simulation-based mastery learning curriculum to teach and assess EM residents' performance of the EDT. METHODS: We developed an EDT curriculum using a mastery learning framework. The minimum passing standard (MPS) for a previously developed 22-item checklist was determined using the Mastery Angoff approach. EM residents at a four-year academic EM residency program underwent baseline testing in performing an EDT on a simulation trainer. Performance was scored by two raters using the checklist. Learners then participated in a novel mastery learning EDT curriculum that included an educational video, hands-on instruction, and deliberate practice. After a three-month period, residents then completed initial post testing. Residents who did not meet the minimum passing standard after post testing participated in additional deliberate practice until mastery was obtained. Baseline and post-test scores, and time to completion of the procedure were compared with paired t-tests. RESULTS: Of 56 eligible EM residents, 54 completed baseline testing. Fifty-two residents completed post-testing until mastery was reached. The minimum passing standard was 91.1%, (21/22 items correct on the checklist). No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS, with deliberate practice sessions not exceeding 40 minutes. Scores from baseline testing to post-testing significantly improved across all postgraduate years from a mean score of 10.2/22 to 21.4/22 (p <0.001). Mean time to complete the procedure improved from baseline testing (6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) to post-testing (5 min 19 seconds, interquartile range 4 min 17sec - 6 min 15 sec; p = 0.001). CONCLUSION: This simulation-based mastery learning curriculum resulted in all residents performing an EDT at a level that met or exceeded the MPS with an overall decrease in time needed to perform the procedure.

Emergency Department Thoracotomy: Development of a Reliable, Validated Checklist for Procedural Training.

OBJECTIVES: Emergency department thoracotomy (EDT) is a rare and challenging procedure. Emergency medicine (EM) residents have limited opportunities to perform the procedure in clinical or educational settings. Standardized, reliable, validated checklists do not exist to evaluate procedural competency. The objectives of this project were twofold: 1) to develop a checklist containing the critical actions for performing an EDT that can be used for future procedural skills training and 2) to evaluate the reliability and validity of the checklist for performing EDT. METHODS: After a literature review, a preliminary 22-item checklist was developed and disseminated to experts in EM and trauma surgery. A modified Delphi method was used to revise the checklist. To assess usability of the checklist, EM and trauma surgery faculty and residents were evaluated performing an EDT while inter-rater reliability was calculated with Cohen's kappa. A Student's t-test was used to compare the performance of participants who had or had not performed a thoracotomy in clinical practice. Item-total correlation was calculated for each checklist item to determine discriminatory ability. RESULTS: A final 22-item checklist was developed for EDT. The overall inter-rater reliability was strong (κ = 0.84) with individual item agreement ranging from moderate to strong (κ = 0.61 to 1.00). Experts (attending physicians and senior residents) performed well on the checklist, achieving an average score of 80% on the checklist. Participants who had performed EDT in clinical practice performed significantly better than those that had not, achieving an average of 80.7% items completed versus 52.3% (p < 0.05). Seventeen of 22 items had an item-total correlation greater than 0.2. CONCLUSIONS: A final 22-item consensus-based checklist was developed for the EDT. Overall inter-rater reliability was strong. This checklist can be used in future studies to serve as a foundation for curriculum development around this important procedure.

The Standardized Letter of Evaluation Narrative: Differences in Language Use by Gender.

INTRODUCTION: Prior research demonstrates gender differences in language used in letters of recommendation. The emergency medicine (EM) Standardized Letter of Evaluation (SLOE) format limits word count and provides detailed instructions for writers. The objective of this study is to examine differences in language used to describe men and women applicants within the SLOE narrative. METHODS: All applicants to a four-year academic EM residency program within a single application year with a first rotation SLOE available were included in the sample. We used the Linguistic Inquiry and Word Count (LIWC) program to analyze word frequency within 16 categories. Descriptive statistics, chi-squared, and t-tests were used to describe the sample; gender differences in word frequency were tested for using Mann-Whitney U tests. RESULTS: Of 1117 applicants to the residency program, 822 (82%) first-rotation SLOEs were available; 64% were men, and 36% were women. We did not find a difference in baseline characteristics including age (mean 27 years), top 25 schools (22.5%), Alpha Omega Alpha Honor Medical Society rates (13%), and having earned advanced degrees (10%). The median word count per SLOE narrative for men was 171 and for women was 180 (p = 0.15). After adjusting for letter length, word frequency differences between genders were only present in two categories: social words (women: 23 words/letter; men: 21 words/letter, p = 0.02) and ability words (women: 2 words/letter; men: 1 word/letter, p = 0.04). We were unable to detect a statistical difference between men and women applicants in the remaining categories, including words representing communal traits, agentic traits, standout adjectives, grindstone traits, teaching words, and research words. CONCLUSION: The small wording differences between genders noted in two categories were statistically significant, but of unclear real-world significance. Future work is planned to evaluate how the SLOE format may contribute to this relative lack of bias compared to other fields and formats.

Immediate and long-term effects of a team-based quality improvement training programme.

BACKGROUND: Although many studies of quality improvement (QI) education programmes report improvement in learners' knowledge and confidence, the impact on learners' future engagement in QI activities is largely unknown and few studies report project measures beyond completion of the programme. METHOD: We developed the Academy for Quality and Safety Improvement (AQSI) to prepare individuals, across multiple departments and professions, to lead QI. The 7-month programme consisted of class work and team-based project work. We assessed participants' knowledge using a multiple choice test and an adapted Quality Improvement Knowledge Assessment Test (QIKAT) before and after the programme. We evaluated participants' postprogramme QI activity and project status using surveys at 6 and 18 months. RESULTS: Over 5 years, 172 individuals and 32 teams participated. Participants had higher multiple choice test (71.9±12.7 vs 79.4±13.2; p<0.001) and adapted QIKAT scores (55.7±16.3 vs 61.8±14.7; p<0.001) after the programme. The majority of participants at 6 months indicated that they had applied knowledge and skills learnt to improve quality in their clinical area (129/148; 87.2%) and to implement QI interventions (92/148; 62.2%). At 18 months, nearly half (48/101; 47.5%) had led other QI projects and many (41/101; 40.6%) had provided QI mentorship to others. Overall, 14 (43.8%) teams had positive postintervention results at AQSI completion and 20 (62.5%) had positive results at some point (ie, completion, 6 months or 18 months after AQSI). CONCLUSIONS: A team-based QI training programme resulted in a high degree of participants' involvement in QI activities beyond completion of the programme. A majority of team projects showed improvement in project measures, often occurring after completion of the programme.

TRIAD VIII: Nationwide Multicenter Evaluation to Determine Whether Patient Video Testimonials Can Safely Help Ensure Appropriate Critical Versus End-of-Life Care.

OBJECTIVE: End-of-life interventions should be predicated on consensus understanding of patient wishes. Written documents are not always understood; adding a video testimonial/message (VM) might improve clarity. Goals of this study were to (1) determine baseline rates of consensus in assigning code status and resuscitation decisions in critically ill scenarios and (2) determine whether adding a VM increases consensus. METHODS: We randomly assigned 2 web-based survey links to 1366 faculty and resident physicians at institutions with graduate medical education programs in emergency medicine, family practice, and internal medicine. Each survey asked for code status interpretation of stand-alone Physician Orders for Life-Sustaining Treatment (POLST) and living will (LW) documents in 9 scenarios. Respondents assigned code status and resuscitation decisions to each scenario. For 1 of 2 surveys, a VM was included to help clarify patient wishes. RESULTS: Response rate was 54%, and most were male emergency physicians who lacked formal advanced planning document interpretation training. Consensus was not achievable for stand-alone POLST or LW documents (68%-78% noted "DNR"). Two of 9 scenarios attained consensus for code status (97%-98% responses) and treatment decisions (96%-99%). Adding a VM significantly changed code status responses by 9% to 62% (P ≤ 0.026) in 7 of 9 scenarios with 4 achieving consensus. Resuscitation responses changed by 7% to 57% (P ≤ 0.005) with 4 of 9 achieving consensus with VMs. CONCLUSIONS: For most scenarios, consensus was not attained for code status and resuscitation decisions with stand-alone LW and POLST documents. Adding VMs produced significant impacts toward achieving interpretive consensus.

A Ten-year Program Evaluation of an Emergency Medicine Scholarly Track in Education Using a Qualitative Approach.

BACKGROUND: Residency scholarly tracks have allowed training programs to better prepare residents for a future in academia. Despite their success, however, few evaluations of these scholarly tracks have been completed to ensure they are meeting the goals of their stakeholders. OBJECTIVE: The objective was to evaluate the education scholarly track at Northwestern University Emergency Medicine by querying key stakeholders on the current state of the track and its ideal state. METHODS: Current emergency medicine residents, faculty, and track alumni were identified as essential stakeholders. Their perspectives on the track were elicited with a focus group and online survey, respectively, and responses were analyzed using grounded theory. RESULTS: Four distinct themes emerged from the analysis as critical to the track: "education skills and learning theory," "education research methodology," "leadership," and "collaboration." CONCLUSION: These themes will be used to inform further development of the track. A similar self-reflection process may benefit other programs with scholarly tracks.

Gender Differences in Language of Standardized Letter of Evaluation Narratives for Emergency Medicine Residency Applicants.

OBJECTIVE: While gender differences in language for letters of recommendation have been identified in other fields, no prior studies have evaluated the narrative portion of the emergency medicine (EM) standardized letter of evaluation (SLOE). We aim to examine the differences in language used to describe male and female applicants within the SLOE narrative. METHODS: Invited applicants to a 4-year academic EM residency program within a single application year with a SLOE were included in the sample. Exclusion criteria were SLOE of applicants from non-Liaison Committee on Medical Education (LCME) schools or first rotation SLOE not available for download. Data were collected on applicant gender, age, rotation grade, Alpha Omega Alpha designation, and medical school rank. The previously validated Linguistic Inquiry and Word Count (LIWC) program was used to analyze frequency of words within categories relevant to letters of recommendation. Descriptive statistics, t-tests, and chi-square tests were employed in analysis. RESULTS: Of 1,025 applicants within a single application year, 265 were invited to interview; 237 applicants had a first rotation SLOE available for analysis. There were no differences between male and female applicants for baseline characteristics. The median word count per SLOE narrative was 199; within the LIWC dictionary and user-defined categories, words within the categories of affiliation and ability appeared more frequently for female applicants. CONCLUSIONS: Our results with respect to the SLOE narrative reinforce prior research that letters of recommendation for female applicants highlight communal characteristics of teamwork, helpfulness, and compassion. Contrary to prior research, ability words highlighting intelligence and skill appeared with greater frequency for female applicants. No pervasive differences were found in other word categories. In this sample, the standardized format of the SLOE resulted in letters that were relatively free of gender bias.

Academic Primer Series: Five Key Papers Fostering Educational Scholarship in Junior Academic Faculty.

INTRODUCTION: Scholarship is an essential part of academic success. Junior faculty members are often unfamiliar with the grounding literature that defines educational scholarship. In this article, the authors aim to summarize five key papers which outline education scholarship in the setting of academic contributions for emerging clinician educators. METHODS: The authors conducted a consensus-building process to generate a list of key papers that describe the importance and significance of academic scholarship, informed by social media sources. They then used a three-round voting methodology, akin to a Delphi study, to determine the most useful papers. RESULTS: A summary of the five most important papers on the topic of academic scholarship, as determined by this mixed group of junior faculty members and faculty developers, is presented in this paper. These authors subsequently wrote a summary of these five papers and discussed their relevance to both junior faculty members and faculty developers. CONCLUSION: Five papers on education scholarship, deemed essential by the authors' consensus process, are presented in this paper. These papers may help provide the foundational background to help junior faculty members gain a grasp of the academic scholarly environment. This list may also inform senior faculty and faculty developers on the needs of junior educators in the nascent stages of their careers.

Workplace Violence and Harassment Against Emergency Medicine Residents.

INTRODUCTION: Several studies have shown that workplace violence in the emergency department (ED) is common. Residents may be among the most vulnerable staff, as they have the least experience with these volatile encounters. The goal for this study was to quantify and describe acts of violence against emergency medicine (EM) residents by patients and visitors and to identify perceived barriers to safety. METHODS: This cross-sectional survey study queried EM residents at multiple New York City hospitals. The primary outcome was the incidence of violence experienced by residents while working in the ED. The secondary outcomes were the subtypes of violence experienced by residents, as well as the perceived barriers to safety while at work. RESULTS: A majority of residents (66%, 78/119) reported experiencing at least one act of physical violence during an ED shift. Nearly all residents (97%, 115/119) experienced verbal harassment, 78% (93/119) had experienced verbal threats, and 52% (62/119) reported sexual harassment. Almost a quarter of residents felt safe "Occasionally," "Seldom" or "Never" while at work. Patient-based factors most commonly cited as contributory to violence included substance use and psychiatric disease. CONCLUSION: Self-reported violence against EM residents appears to be a significant problem. Incidence of violence and patient risk factors are similar to what has been found previously for other ED staff. Understanding the prevalence of workplace violence as well as the related systems, environmental, and patient-based factors is essential for future prevention efforts.